Ultrapotent Broadly Neutralizing Human-llama Bispecific Antibodies against HIV-1.

Broadly neutralizing antibodies are proposed as therapeutic and prophylactic agents against HIV-1, but their potency and breadth are less than optimal. This study describes the immunization of a llama with the prefusion-stabilized HIV-1 envelope (Env) trimer, BG505 DS-SOSIP, and the identification and improvement of potent neutralizing nanobodies recognizing the CD4-binding site (CD4bs) of vulnerability. Two of the vaccine-elicited CD4bs-targeting nanobodies, G36 and R27, when engineered into a triple tandem format with llama IgG2a-hinge region and human IgG1-constant region (G36×3-IgG2a and R27×3-IgG2a), neutralized 96% of a multiclade 208-strain panel at geometric mean IC80s of 0.314 and 0.033 µg mL-1, respectively. Cryo-EM structures of these nanobodies in complex with Env trimer revealed the two nanobodies to neutralize HIV-1 by mimicking the recognition of the CD4 receptor. To enhance their neutralizing potency and breadth, nanobodies are linked to the light chain of the V2-apex-targeting broadly neutralizing antibody, CAP256V2LS. The resultant human-llama bispecific antibody CAP256L-R27×3LS exhibited ultrapotent neutralization and breadth exceeding other published HIV-1 broadly neutralizing antibodies, with pharmacokinetics determined in FcRn-Fc mice similar to the parent CAP256V2LS. Vaccine-elicited llama nanobodies, when combined with V2-apex broadly neutralizing antibodies, may therefore be able to fulfill anti-HIV-1 therapeutic and prophylactic clinical goals.

Cleavage-intermediate Lassa virus trimer elicits neutralizing responses, identifies neutralizing nanobodies, and reveals an apex-situated site-of-vulnerability.

Lassa virus (LASV) infection is expanding outside its traditionally endemic areas in West Africa, posing a pandemic biothreat. LASV-neutralizing antibodies, moreover, have proven difficult to elicit. To gain insight into LASV neutralization, here we develop a prefusion-stabilized LASV glycoprotein trimer (GPC), pan it against phage libraries comprising single-domain antibodies (nanobodies) from shark and camel, and identify one, D5, which neutralizes LASV. Cryo-EM analyses reveal D5 to recognize a cleavage-dependent site-of-vulnerability at the trimer apex. The recognized site appears specific to GPC intermediates, with protomers lacking full cleavage between GP1 and GP2 subunits. Guinea pig immunizations with the prefusion-stabilized cleavage-intermediate LASV GPC, first as trimer and then as a nanoparticle, induce neutralizing responses, targeting multiple epitopes including that of D5; we identify a neutralizing antibody (GP23) from the immunized guinea pigs. Collectively, our findings define a prefusion-stabilized GPC trimer, reveal an apex-situated site-of-vulnerability, and demonstrate elicitation of LASV-neutralizing responses by a cleavage-intermediate LASV trimer.

Boron-intercalation-triggered crystalline transition of Pd nanosheet assemblies for an enhanced oxygen reduction reaction.

The development of effective and stable cathode electrocatalysts is highly desired for fuel cells. Controlling the composition and morphology of Pd-based materials can provide a great opportunity to improve their oxygen reduction reaction (ORR) performance. Here, we report the synthesis of hexagonal close-packed (hcp) Pd2B nanosheet assemblies (Pd2B NAs) via the boronation reaction between as-synthesized Pd NAs and N,N-dimethylformamide. The hcp Pd2B NAs with uniform pore distribution can provide sufficient active sites for ORRs. The insertion of B atoms can induce the phase transition from face-centered cubic structure to hcp structure, as the most thermodynamically stable phase in the Pd-B alloy, which is beneficial for enhancing the ORR stability and toxicity resistance. Therefore, the hcp Pd2B NAs exhibit superior mass activity, specific activity and excellent stability for ORR. The present strategy of boron-intercalation-triggered crystalline transition of Pd-based nanomaterials is valuable for the design of metal-nonmetal catalysts with enhanced performance.

Human CD4-Binding Site Antibody Elicited by Polyvalent DNA Prime-Protein Boost Vaccine Neutralizes Cross-Clade Tier-2-HIV Strains.

The vaccine elicitation of HIV-neutralizing antibodies with tier-2-neutralization breadth has been a challenge. Here, we report the isolation and characteristics of a CD4-binding site specific monoclonal antibody, HmAb64, from a human volunteer immunized with a polyvalent gp120 DNA prime-protein boost vaccine. HmAb64 derived from heavy chain variable germline gene IGHV1-18, light chain germline gene IGKV1-39, and had a 3rd heavy chain complementarity determining region (CDR H3) of 15 amino acids. On a cross-clade panel of 208 HIV-1 pseudo-virus strains, HmAb64 neutralized 21 (10%), including tier-2 neutralization resistant strains from clades B, BC, C, and G. The cryo-EM structure of the antigen-binding fragment of HmAb64 bound to a conformation between prefusion closed and occluded open forms of envelope trimer, using both heavy and light CDR3s to recognize the CD4-binding loop, a critical component of the CD4-binding site. A gp120 subunit-based vaccine can thus elicit an antibody capable of tier 2-HIV neutralization.

Structure-based design of a single-chain triple-disulfide-stabilized fusion-glycoprotein trimer that elicits high-titer neutralizing responses against human metapneumovirus.

The Pneumoviridae family of viruses includes human metapneumovirus (HMPV) and respiratory syncytial virus (RSV). The closely related Paramyxoviridae family includes parainfluenza viruses (PIVs). These three viral pathogens cause acute respiratory tract infections with substantial disease burden in the young, the elderly, and the immune-compromised. While promising subunit vaccines are being developed with prefusion-stabilized forms of the fusion glycoproteins (Fs) of RSV and PIVs, for which neutralizing titers elicited by the prefusion (pre-F) conformation of F are much higher than for the postfusion (post-F) conformation, with HMPV, pre-F and post-F immunogens described thus far elicit similar neutralizing responses, and it has been unclear which conformation, pre-F or post-F, would be the most effective HMPV F-vaccine immunogen. Here, we investigate the impact of further stabilizing HMPV F in the pre-F state. We replaced the furin-cleavage site with a flexible linker, creating a single chain F that yielded increased amounts of pre-F stabilized trimers, enabling the generation and assessment of F trimers stabilized by multiple disulfide bonds. Introduced prolines could increase both expression yields and antigenic recognition by the pre-F specific antibody, MPE8. The cryo-EM structure of a triple disulfide-stabilized pre-F trimer with the variable region of antibody MPE8 at 3.25-Å resolution confirmed the formation of designed disulfides and provided structural details on the MPE8 interface. Immunogenicity assessments in naïve mice showed the triple disulfide-stabilized pre-F trimer could elicit high titer neutralization, >10-fold higher than elicited by post-F. Immunogenicity assessments in pre-exposed rhesus macaques showed the triple disulfide-stabilized pre-F could recall high neutralizing titers after a single immunization, with little discrimination in the recall response between pre-F and post-F immunogens. However, the triple disulfide-stabilized pre-F adsorbed HMPV-directed responses from commercially available pooled human immunoglobulin more fully than post-F. Collectively, these results suggest single-chain triple disulfide-stabilized pre-F trimers to be promising HMPV-vaccine antigens.

Soluble prefusion-closed HIV-envelope trimers with glycan-covered bases.

Soluble HIV-1-envelope (Env) trimers elicit immune responses that target their solvent-exposed protein bases, the result of removing these trimers from their native membrane-bound context. To assess whether glycosylation could limit these base responses, we introduced sequons encoding potential N-linked glycosylation sites (PNGSs) into base-proximal regions. Expression and antigenic analyses indicated trimers bearing six-introduced PNGSs to have reduced base recognition. Cryo-EM analysis revealed trimers with introduced PNGSs to be prone to disassembly and introduced PNGS to be disordered. Protein-base and glycan-base trimers induced reciprocally symmetric ELISA responses, in which only a small fraction of the antibody response to glycan-base trimers recognized protein-base trimers and vice versa. EM polyclonal epitope mapping revealed glycan-base trimers -even those that were stable biochemically- to elicit antibodies that recognized disassembled trimers. Introduced glycans can thus mask the protein base but their introduction may yield neo-epitopes that dominate the immune response.

HIV-1 neutralizing antibodies elicited in humans by a prefusion-stabilized envelope trimer form a reproducible class targeting fusion peptide.

Elicitation of antibodies that neutralize the tier-2 neutralization-resistant isolates that typify HIV-1 transmission has been a long-sought goal. Success with prefusion-stabilized envelope trimers eliciting autologous neutralizing antibodies has been reported in multiple vaccine-test species, though not in humans. To investigate elicitation of HIV-1 neutralizing antibodies in humans, here, we analyze B cells from a phase I clinical trial of the "DS-SOSIP"-stabilized envelope trimer from strain BG505, identifying two antibodies, N751-2C06.01 and N751-2C09.01 (named for donor-lineage.clone), that neutralize the autologous tier-2 strain, BG505. Though derived from distinct lineages, these antibodies form a reproducible antibody class that targets the HIV-1 fusion peptide. Both antibodies are highly strain specific, which we attribute to their partial recognition of a BG505-specific glycan hole and to their binding requirements for a few BG505-specific residues. Prefusion-stabilized envelope trimers can thus elicit autologous tier-2 neutralizing antibodies in humans, with initially identified neutralizing antibodies recognizing the fusion-peptide site of vulnerability.

RNA helicase DDX6 and scaffold protein GW182 in P-bodies promote biogenesis of stress granules.

Two prominent cytoplasmic RNA granules, ubiquitous RNA-processing bodies (PB) and inducible stress granules (SG), regulate mRNA translation and are intimately related. In this study, we found that arsenite (ARS)-induced SG formed in a stepwise process is topologically and mechanically linked to PB. Two essential PB components, GW182 and DDX6, are repurposed under stress to play direct but distinguishable roles in SG biogenesis. By providing scaffolding activities, GW182 promotes the aggregation of SG components to form SG bodies. DEAD-box helicase DDX6 is also essential for the proper assembly and separation of PB from SG. DDX6 deficiency results in the formation of irregularly shaped 'hybrid' PB/SG granules with accumulated components of both PB and SG. Wild-type DDX6, but not its helicase mutant E247A, can rescue the separation of PB from SG in DDX6KO cells, indicating a requirement of DDX6 helicase activity for this process. DDX6 activity in biogenesis of both PB and SG in the cells under stress is further modulated by its interaction with two protein partners, CNOT1 and 4E-T, of which knockdown affects the formation of both PB and also SG. Together, these data highlight a new functional paradigm between PB and SG biogenesis during the stress.

Enhanced evasion of neutralizing antibody response by Omicron XBB.1.5, CH.1.1, and CA.3.1 variants.

Omicron subvariants continuingly challenge current vaccination strategies. Here, we demonstrate nearly complete escape of the XBB.1.5, CH.1.1, and CA.3.1 variants from neutralizing antibodies stimulated by three doses of mRNA vaccine or by BA.4/5 wave infection, but neutralization is rescued by a BA.5-containing bivalent booster. CH.1.1 and CA.3.1 show strong immune escape from monoclonal antibody S309. Additionally, XBB.1.5, CH.1.1, and CA.3.1 spike proteins exhibit increased fusogenicity and enhanced processing compared with BA.2. Homology modeling reveals the key roles of G252V and F486P in the neutralization resistance of XBB.1.5, with F486P also enhancing receptor binding. Further, K444T/M and L452R in CH.1.1 and CA.3.1 likely drive escape from class II neutralizing antibodies, whereas R346T and G339H mutations could confer the strong neutralization resistance of these two subvariants to S309-like antibodies. Overall, our results support the need for administration of the bivalent mRNA vaccine and continued surveillance of Omicron subvariants.

Diverse Murine Vaccinations Reveal Distinct Antibody Classes to Target Fusion Peptide and Variation in Peptide Length to Improve HIV Neutralization.

While neutralizing antibodies that target the HIV-1 fusion peptide have been elicited in mice by vaccination, antibodies reported thus far have been from only a single antibody class that could neutralize ~30% of HIV-1 strains. To explore the ability of the murine immune system to generate cross-clade neutralizing antibodies and to investigate how higher breadth and potency might be achieved, we tested 17 prime-boost regimens that utilized diverse fusion peptide-carrier conjugates and HIV-1 envelope trimers with different fusion peptides. We observed priming in mice with fusion peptide-carrier conjugates of variable peptide length to elicit higher neutralizing responses, a result we confirmed in guinea pigs. From vaccinated mice, we isolated 21 antibodies, belonging to 4 distinct classes of fusion peptide-directed antibodies capable of cross-clade neutralization. Top antibodies from each class collectively neutralized over 50% of a 208-strain panel. Structural analyses - both X-ray and cryo-EM - revealed each antibody class to recognize a distinct conformation of fusion peptide and to have a binding pocket capable of accommodating diverse fusion peptides. Murine vaccinations can thus elicit diverse neutralizing antibodies, and altering peptide length during prime can improve the elicitation of cross-clade responses targeting the fusion peptide site of HIV-1 vulnerability. IMPORTANCE The HIV-1 fusion peptide has been identified as a site for elicitation of broadly neutralizing antibodies, with prior studies demonstrating that priming with fusion peptide-based immunogens and boosting with soluble envelope (Env) trimers can elicit cross-clade HIV-1-neutralizing responses. To improve the neutralizing breadth and potency of fusion peptide-directed responses, we evaluated vaccine regimens that incorporated diverse fusion peptide-conjugates and Env trimers with variation in fusion peptide length and sequence. We found that variation in peptide length during prime elicits enhanced neutralizing responses in mice and guinea pigs. We identified vaccine-elicited murine monoclonal antibodies from distinct classes capable of cross-clade neutralization and of diverse fusion peptide recognition. Our findings lend insight into improved immunogens and regimens for HIV-1 vaccine development.

An influenza H1 hemagglutinin stem-only immunogen elicits a broadly cross-reactive B cell response in humans.

Current yearly seasonal influenza vaccines primarily induce an antibody response directed against the immunodominant but continually diversifying hemagglutinin (HA) head region. These antibody responses provide protection against the vaccinating strain but little cross-protection against other influenza strains or subtypes. To focus the immune response on subdominant but more conserved epitopes on the HA stem that might protect against a broad range of influenza strains, we developed a stabilized H1 stem immunogen lacking the immunodominant head displayed on a ferritin nanoparticle (H1ssF). Here, we evaluated the B cell response to H1ssF in healthy adults ages 18 to 70 in a phase 1 clinical trial (NCT03814720). We observed both a strong plasmablast response and sustained elicitation of cross-reactive HA stem-specific memory B cells after vaccination with H1ssF in individuals of all ages. The B cell response was focused on two conserved epitopes on the H1 stem, with a highly restricted immunoglobulin repertoire unique to each epitope. On average, two-thirds of the B cell and serological antibody response recognized a central epitope on the H1 stem and exhibited broad neutralization across group 1 influenza virus subtypes. The remaining third recognized an epitope near the viral membrane anchor and was largely limited to H1 strains. Together, we demonstrate that an H1 HA immunogen lacking the immunodominant HA head produces a robust and broadly neutralizing HA stem-directed B cell response.

Effect of Passive Administration of Monoclonal Antibodies Recognizing Simian Immunodeficiency Virus (SIV) V2 in CH59-Like Coil/Helical or ß-Sheet Conformations on Time of SIVmac251 Acquisition.

The monoclonal antibodies (MAbs) NCI05 and NCI09, isolated from a vaccinated macaque that was protected from multiple simian immunodeficiency virus (SIV) challenges, both target an overlapping, conformationally dynamic epitope in SIV envelope variable region 2 (V2). Here, we show that NCI05 recognizes a CH59-like coil/helical epitope, whereas NCI09 recognizes a ß-hairpin linear epitope. In vitro, NCI05 and, to a lesser extent, NCI09 mediate the killing of SIV-infected cells in a CD4-dependent manner. Compared to NCI05, NCI09 mediates higher titers of antibody-dependent cellular cytotoxicity (ADCC) to gp120-coated cells, as well as higher levels of trogocytosis, a monocyte function that contributes to immune evasion. We also found that passive administration of NCI05 or NCI09 to macaques did not affect the risk of SIVmac251 acquisition compared to controls, demonstrating that these anti-V2 antibodies alone are not protective. However, NCI05 but not NCI09 mucosal levels strongly correlated with delayed SIVmac251 acquisition, and functional and structural data suggest that NCI05 targets a transient state of the viral spike apex that is partially opened, compared to its prefusion-closed conformation. IMPORTANCE Studies suggest that the protection against SIV/simian-human immunodeficiency virus (SHIV) acquisition afforded by the SIV/HIV V1 deletion-containing envelope immunogens, delivered by the DNA/ALVAC vaccine platform, requires multiple innate and adaptive host responses. Anti-inflammatory macrophages and tolerogenic dendritic cells (DC-10), together with CD14+ efferocytes, are consistently found to correlate with a vaccine-induced decrease in the risk of SIV/SHIV acquisition. Similarly, V2-specific antibody responses mediating ADCC, Th1 and Th2 cells expressing no or low levels of CCR5, and envelope-specific NKp44+ cells producing interleukin 17 (IL-17) also are reproducible correlates of decreased risk of virus acquisition. We focused on the function and the antiviral potential of two monoclonal antibodies (NCI05 and NCI09) isolated from vaccinated animals that differ in antiviral function in vitro and recognize V2 in a linear (NCI09) or coil/helical (NCI05) conformation. We demonstrate that NCI05, but not NCI09, delays SIVmac251 acquisition, highlighting the complexity of antibody responses to V2.

Rhesus macaque Bcl-6/Bcl-xL B cell immortalization: Discovery of HIV-1 neutralizing antibodies from lymph node.

Many HIV-1 vaccines are designed to elicit neutralizing antibodies, and pre-clinical testing is often carried out in rhesus macaques (RMs). We have therefore adapted a method of B cell immortalization for use with RM B cells. In this system, RM B cells are activated with CD40 ligand and RM IL-21 before transduction with a retroviral vector encoding Bcl-6, Bcl-xL, and green fluorescent protein. Importantly, RM B cells from lymph nodes are more effectively immortalized by this method than B cells from PBMC, a difference not seen in humans. We suggest the discrepancy between these two tissues is due to increased expression of CD40 on RM lymph node B cells. Immortalized RM B cells expand long-term, undergo minimal somatic hypermutation, express surface B cell receptor, and secrete antibodies into culture. This allows for the identification of cells based on antigen specificity and/or functional assays. Here, we show the characterization of this system and its application for the isolation of HIV-1 neutralizing antibodies from a SHIV.CH505-infected animal, both with and without antigen probe. Taken together, we show that Bcl-6/xL immortalization is a valuable and flexible tool for antibody discovery in RMs, but with important distinctions from application of the system in human cells.

Enhancing Chinese Character Representation With Lattice-Aligned Attention.

Word-character lattice models have been proved to be effective for some Chinese natural language processing (NLP) tasks, in which word boundary information is fused into character sequences. However, due to the inherently unidirectional sequential nature, prior approaches have only learned sequential interactions of character-word instances but fail to capture fine-grained correlations in word-character spaces. In this article, we propose a lattice-aligned attention network (LAN) that aims to model dense interactions over word-character lattice structure for enhancing character representations. By carefully combining cross-lattice module, gated word-character semantic fusion unit, and self-lattice attention module, the network can explicitly capture fine-grained correlations across different spaces (e.g., word-to-character and character-to-character), thus significantly improving model performance. Experimental results on three Chinese NLP benchmark tasks demonstrate that LAN obtains state-of-the-art results compared to several competitive approaches.

Extraordinary Titer and Broad Anti-SARS-CoV-2 Neutralization Induced by Stabilized RBD Nanoparticles from Strain BA.5.

The receptor-binding domain (RBD) of the SARS-CoV-2 spike is a primary target of neutralizing antibodies and a key component of licensed vaccines. Substantial mutations in RBD, however, enable current variants to escape immunogenicity generated by vaccination with the ancestral (WA1) strain. Here, we produce and assess self-assembling nanoparticles displaying RBDs from WA1 and BA.5 strains by using the SpyTag:SpyCatcher system for coupling. We observed both WA1- and BA.5-RBD nanoparticles to degrade substantially after a few days at 37 °C. Incorporation of nine RBD-stabilizing mutations, however, increased yield ~five-fold and stability such that more than 50% of either the WA1- or BA.5-RBD nanoparticle was retained after one week at 37 °C. Murine immunizations revealed that the stabilized RBD-nanoparticles induced ~100-fold higher autologous neutralization titers than the prefusion-stabilized (S2P) spike at a 2 µg dose. Even at a 25-fold lower dose where S2P-induced neutralization titers were below the detection limit, the stabilized BA.5-RBD nanoparticle induced homologous titers of 12,795 ID50 and heterologous titers against WA1 of 1767 ID50. Assessment against a panel of ß-coronavirus variants revealed both the stabilized BA.5-RBD nanoparticle and the stabilized WA1-BA.5-(mosaic)-RBD nanoparticle to elicit much higher neutralization breadth than the stabilized WA1-RBD nanoparticle. The extraordinary titer and high neutralization breadth elicited by stabilized RBD nanoparticles from strain BA.5 make them strong candidates for next-generation COVID-19 vaccines.

Application of B cell immortalization for the isolation of antibodies and B cell clones from vaccine and infection settings.

The isolation and characterization of neutralizing antibodies from infection and vaccine settings informs future vaccine design, and methodologies that streamline the isolation of antibodies and the generation of B cell clones are of great interest. Retroviral transduction to express Bcl-6 and Bcl-xL and transform primary B cells has been shown to promote long-term B cell survival and antibody secretion in vitro, and can be used to isolate antibodies from memory B cells. However, application of this methodology to B cell subsets from different tissues and B cells from chronically infected individuals has not been well characterized. Here, we characterize Bcl-6/Bcl-xL B cell immortalization across multiple tissue types and B cell subsets in healthy and HIV-1 infected individuals, as well as individuals recovering from malaria. In healthy individuals, naïve and memory B cell subsets from PBMCs and tonsil tissue transformed with similar efficiencies, and displayed similar characteristics with respect to their longevity and immunoglobulin secretion. In HIV-1-viremic individuals or in individuals with recent malaria infections, the exhausted CD27-CD21- memory B cells transformed with lower efficiency, but the transformed B cells expanded and secreted IgG with similar efficiency. Importantly, we show that this methodology can be used to isolate broadly neutralizing antibodies from HIV-infected individuals. Overall, we demonstrate that Bcl-6/Bcl-xL B cell immortalization can be used to isolate antibodies and generate B cell clones from different B cell populations, albeit with varying efficiencies.

Cryo-EM structures of prefusion SIV envelope trimer.

Simian immunodeficiency viruses (SIVs) are lentiviruses that naturally infect non-human primates of African origin and seeded cross-species transmissions of HIV-1 and HIV-2. Here we report prefusion stabilization and cryo-EM structures of soluble envelope (Env) trimers from rhesus macaque SIV (SIVmac) in complex with neutralizing antibodies. These structures provide residue-level definition for SIV-specific disulfide-bonded variable loops (V1 and V2), which we used to delineate variable-loop coverage of the Env trimer. The defined variable loops enabled us to investigate assembled Env-glycan shields throughout SIV, which we found to comprise both N- and O-linked glycans, the latter emanating from V1 inserts, which bound the O-link-specific lectin jacalin. We also investigated in situ SIVmac-Env trimers on virions, determining cryo-electron tomography structures at subnanometer resolutions for an antibody-bound complex and a ligand-free state. Collectively, these structures define the prefusion-closed structure of the SIV-Env trimer and delineate variable-loop and glycan-shielding mechanisms of immune evasion conserved throughout SIV evolution.

Co-immunization with hemagglutinin stem immunogens elicits cross-group neutralizing antibodies and broad protection against influenza A viruses.

Current influenza vaccines predominantly induce immunity to the hypervariable hemagglutinin (HA) head, requiring frequent vaccine reformulation. Conversely, the immunosubdominant yet conserved HA stem harbors a supersite that is targeted by broadly neutralizing antibodies (bnAbs), representing a prime target for universal vaccines. Here, we showed that the co-immunization of two HA stem immunogens derived from group 1 and 2 influenza A viruses elicits cross-group protective immunity and neutralizing antibody responses in mice, ferrets, and nonhuman primates (NHPs). Immunized mice were protected from multiple group 1 and 2 viruses, and all animal models showed broad serum-neutralizing activity. A bnAb isolated from an immunized NHP broadly neutralized and protected against diverse viruses, including H5N1 and H7N9. Genetic and structural analyses revealed strong homology between macaque and human bnAbs, illustrating common biophysical constraints for acquiring cross-group specificity. Vaccine elicitation of stem-directed cross-group-protective immunity represents a step toward the development of broadly protective influenza vaccines.

Query-Adaptive Late Fusion for Hierarchical Fine-Grained Video-Text Retrieval.

Recently, a hierarchical fine-grained fusion mechanism has been proved effective in cross-modal retrieval between videos and texts. Generally, the hierarchical fine-grained semantic representations (video-text semantic matching is decomposed into three levels including global-event representation matching, action-relation representation matching, and local-entity representation matching) to be fused can work well by themselves for the query. However, in real-world scenarios and applications, existing methods failed to adaptively estimate the effectiveness of multiple levels of the semantic representations for a given query in advance of multilevel fusion, resulting in a worse performance than expected. As a result, it is extremely essential to identify the effectiveness of hierarchical semantic representations in a query-adaptive manner. To this end, this article proposes an effective query-adaptive multilevel fusion (QAMF) model based on manipulating multiple similarity scores between the hierarchical visual and text representations. First, we decompose video-side and text-side representations into hierarchical semantic representations consisting of global-event level, action-relation level, and local-entity level, respectively. Then, the multilevel representation of the video-text pair is aligned to calculate the similarity score for each level. Meanwhile, the sorted similarity score curves of the good semantic representation are different from the inferior ones, which exhibit a "cliff" shape and gradually decline (see Fig. fig1 as an example). Finally, we leverage the Gaussian decay function to fit the tail of the score curve and calculate the area under the normalized sorted similarity curve as the indicator of semantic representation effectiveness, namely, the area of good semantic representation is small, and vice versa. Extensive experiments on three public benchmark video-text datasets have demonstrated that our method consistently outperforms the state-of-the-art (SoTA). A simple demo of QAMF will soon be publicly available on our homepage: https://github.com/Lab-ANT.